Introduction to HPC
Scientific Programming with Python
Manuel Holtgrewe 
Berlin Institute of Health at Charité
Introduction
- Jupyter Notebooks are a web-based interactive computing environment
- Jupyter Lab is the next generation
- You edit cells with python code which is executed in a kernel
- It’s best explained by example ;-)
Using Jupyter (1)
- There is no magic!
- Packages can be installed in the conda environment (or via pip)
mamba install -y plotly_express
- The kernel keeps the current state
- To ensure that you have a clean state
- restart kernel and rerun all cells
- Bonus:
ipythonis a turbo-charged interpreter on the command line - Useful: Markdown cells for documentation
Data Visualization (1)
- Plotly Express is a high-level plotting library for Python
- It is based on Plotly which is more low-level
- This is really nice to create interactive plots!
- Suggested exercise for later:
- Go through Plotly Express Tutorial
- Again, try it in Jupyter via OnDemand Portal
Data Visualization (2)
- Vega is a declarative language for data visualization
- This is more advanced but also very powerful
Data Visualization (3)
- Bokeh is a Python library for interactive data visualization
- An alternative to Plotly Express
- Here are the official tutorials
Estimator Cheat Sheet
Example: Clustering
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
# Generate synthetic data
data, _ = make_blobs(n_samples=300, centers=4, random_state=42)
# Create K-Means model
kmeans = KMeans(n_clusters=4)
# Fit the model to the data
kmeans.fit(data)
# Predict clusters for each data point
labels = kmeans.labels_
# Get cluster centers
cluster_centers = kmeans.cluster_centers_
plt.scatter(data[:, 0], data[:, 1], c=labels, cmap='viridis')
plt.scatter(cluster_centers[:, 0], cluster_centers[:, 1], c='red', marker='x', s=200)
plt.title('K-Means Clustering')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.show()
Example: Regression
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
# Generate synthetic regression data
X, y = make_regression(n_samples=100, n_features=1, noise=10, random_state=42)
# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Create Linear Regression model
model = LinearRegression()
# Fit the model to the training data
model.fit(X_train, y_train)
# Make predictions on the test data
y_pred = model.predict(X_test)
# Calculate Mean Squared Error
mse = mean_squared_error(y_test, y_pred)
# Visualize the data and regression line
plt.scatter(X_test, y_test, label='Test Data')
plt.plot(X_test, y_pred, color='red', label='Regression Line')
plt.title(f'Linear Regression (MSE: {mse:.2f})')
plt.xlabel('Feature')
plt.ylabel('Target')
plt.legend()
plt.show()
Example: Classification
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix
# Generate synthetic classification data
X, y = make_classification(n_samples=200, n_features=2, n_informative=2,
n_redundant=0, n_clusters_per_class=1, random_state=42)
# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Create Logistic Regression model
model = LogisticRegression()
# Fit the model to the training data
model.fit(X_train, y_train)
# Make predictions on the test data
y_pred = model.predict(X_test)
# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
# Create a confusion matrix
conf_matrix = confusion_matrix(y_test, y_pred)
# Visualize the data and decision boundary
plt.scatter(X_test[:, 0], X_test[:, 1], c=y_test, cmap='coolwarm', marker='o', label='True Class')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
# Create a mesh to plot decision boundary
x_min, x_max = X_test[:, 0].min() - 1, X_test[:, 0].max() + 1
y_min, y_max = X_test[:, 1].min() - 1, X_test[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.01), np.arange(y_min, y_max, 0.01))
Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, alpha=0.4, cmap='coolwarm')
plt.title(f'Logistic Regression (Accuracy: {accuracy:.2f})')
plt.legend()
plt.show()
Introduction
- TensorFlow: popular open-source library for machine learning, by Google
- main features:
- high-performance numerical computations
- deep learning
- easy of use, excellent documentation
- applications: “you name it”, e.g.
- image classification
- bioinformatics
